A Central Limit Theorem, 

and related results, 

. for a two-color randomly reinforced urn 

o ■ 

CN ; Giacomo Aletti*, Caterina May*^ and Piercesare Secchi'^ 

u '• 

April 22, 2009 

(N 
(N 

*Dipartimento di Matematica "F. Enrigues" 
' Universita degli Studi di Milano 

. via Cesare Saldini 50, 20133 Milano, Italy 

fH [ giacomo.aletti@mat.unimi.it 

^ '• ^Dipartimento SEMEQ 

Universita del Piemonte Orientale 
via Perrone 18, 28100 Novara, Italy 
Tlj" ■ caterina.may@eco.unipmn.it 

I ^ MOX-Dipartimento di Matematica 

' Politecnico di Milano 

. via Bonardi 9, 20133 Milano, Italy 

piercesare . secchi@polimi . it 



OO 
O 



Keywords: reinforced processes, generalized Polya urns, convergence of 
conditional distributions. 



^ ; 2000 AMS Subject Classification: 60F05 



Abstract 

We prove a Central Limit Theorem for the sequence of random 
compositions of a two-color randomly reinforced urn. As a conse- 
quence, we are able to show that the distribution of the urn limit 
composition has no point masses. 
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1 Introduction 



Consider an urn containing initially x balls of color black and y balls of color 
white, with x and y non negative real numbers such that x + y > 0. The urn 
is sequentially sampled: whenever the color of the sampled ball is black, the 
ball is replaced in the urn together with a random number of black balls, 
generated at that instant from a distribution n with non negative bounded 
support; whenever the sampled ball is white, the ball is replaced in the urn 
together with a random number of balls, generated at that instant from 
a distribution u with non negative bounded support. This is an informal 
description of the Randomly Reinforced Urn (RRU) introduced in [13] and 
studied in [H [21 El El El CHI HI] under various assumptions concerning the 
reinforcement distributions fi and u. The urn has an interesting potential 
for applications since it describes a general model for reinforcement learning 
([21 [8]); in clinical trials, it implements an optimal response adaptive design 

([si[ini[i2i[is]). 

The focus of this paper is on the asymptotic behavior of the sequence 
{Zn} describing the random proportions of black balls in the urn along the 
sampling sequence; in [13] it is proved that the sequence {Zn} converges 
almost surely to a random limit Zoo £ [0, !]• 

When fi = u, a. RRU is a special case of the generalized Polya urn 
studied by Crimaldi in [3]; for the sequence of random proportions {Zn} 
generated by her urn, Crimaldi proves a Central Limit Theorem by showing 
almost sure conditional convergence to a Gaussian kernel of the sequence 
{y/n{Zn — Zoo)}. Crimaldi's result does not hold for a general RRU; in 
this paper we extend it to cover the case of a RRU with reinforcement 
distributions fi and u having the same mean. When the means of fi and 
u are different, the limit proportion Zoo of a RRU is a point mass either 
in 1 or in 0, according to the reinforcement distribution having the larger 
mean, as proved with different arguments in [21 El [13] • 

A nice implication of our RRU Central Limit Theorem is that we are 
now able to prove that the distribution of the limit proportion Zoo has no 
point masses in [0, 1], when the means of the reinforcement distributions are 
the same. This gives a new drive to the problem concerning the absolute 
continuity of the distribution of the limit proportion of a generalized Polya 
urn, considered, for instance, also in [T6j . 

The paper is organized as follows. In the next section we will formally 
introduce the RRU model along with the notation used in the paper. The 
main results of the paper are stated in section 3, while proofs appear in the 
following section. A remark on the absolute continuity of the distribution 
of Zoo concludes the paper. 
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2 Model description and notations 



On a rich enough probabihty space {fl,A,V), define two independent in- 
finite sequences of random elements, {Un} and W^n)}; {Un} is a se- 
quence of i.i.d. random variables uniformly distributed on [0,1], while 
{{Vn, W^n)} is a sequence of i.i.d bivariate random vectors with components 
uniformly distributed on [0,1]. Given two probability distributions fi and 
u on [0, /?], with P > 0, indicate their quantile functions with and g^, re- 
spectively. Then, define an infinite sequence {{Rx{n), Ryin))} of bivariate 
random vectors by setting, for all n, 

Rx{n) = Qf^iVn) and Ryin) = q^{Wn). 

Note that, whereas the sequences {{Rx{n), Ryin))} and {Un} are indepen- 
dent, the random variables Rxin) and Ry{n) might be dependent; how- 
ever, for every n, their distributions are /i and z/, respectively. We indicate 
with and m^, and with and a^, the means, and the variances, of 
two random variables Rx and Ry having probability distributions /i and 
respectively. 

We are now ready to introduce a process whose law is that of a Ran- 
domly Reinforced Urn as defined in [13] . Let x and y be two non-negative 
real numbers such that x+y > 0. Set Xq = x,Yo = y, and, for n = 0, 1, 2, 
let 

f Xn+l = + Rxin + l)6n+l, . x 

\ = Yn + Ry{n + l){l-6n+i), ^ ' 

where the variable 6n+i is the indicator of the event {f/„+i < X„(X„ + 
Yn)^^}. The law of {(X„,y„,)} is that of the stochastic process counting, 
along the sampling sequence, the number of black and white balls present 
in a RRU with initial composition [x, y) and reinforcement distributions 
equal to and u, respectively. 
Forn = 0, 1,2, ... let 

7 — '^'^ 
" ~ Y -u V ' 

Zn represents the proportion of black balls in the urn before the (n -|- 1)- 
th ball is sampled from it. In [H] it is proved that {Zn} is eventually a 
bounded sub- or super-martingale, according to the mean of fi being larger 
or smaller than that of u. Hence, for n growing to infinity, Zn converges 
almost surely, and in L^, 1 < p < oo, to a random variable Z^o € [0, 1]. 
For n = 1,2,..., let Rn = 6nRx{n) + (1 — 6n)Ry{n) be the urn reinforce- 
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ment, when the urn is sampled for the n-th time, and set 
X ^ Rxin) Y ^ Ryin) 

Qn-1 = "^^H ^ — ^nQn-1 + ~ ^n)Qn-l^ 
l^i=l 

with = = Qn-1 = 1 if = for all 2 = For 

shortness, we will write Dn for the random number X„ + y„, interpreted 
as the size of the urn before it is sampled for the (n + l)-th time. Clearly, 
Dq = X + y while -D^+i = -D„ + Rn+i, for n = 0, 1, 2, .... Finally let An = 
<j{Ui, . . . , Un, (Vi, Wi), . . . , {Vn, Wn)) and consider the filtration {A„}; for 
77, = 1, 2, we indicate with M„ and An the two terms given by the Doob's 
semi-martingale decomposition of i.e. 

Zn = Zo + Mn + An, 

where {M„} is a zero mean martingale with respect to {^n}, while {An} 
is previsible with respect to {^n}- Theorem 2 in [H] shows that {An} is 
eventually increasing or decreasing. 



3 Main results 

For every set A E A, every u E Q and n = 1, 2, define 

Kn{uJ,A) = P{V^{Zn - ^oo) e A\An){uj); 

i.e. -fCn is a version of the conditional distribution of ^/n{Zn — Zoo) given 
When the reinforcement distributions of an RRU are the same, i.e. /i = i^, 
and /i is different from the point mass at 0, Corollary 4.1 in [1] shows that, 
for almost every G f2, the sequence of probability distribution {K^iu), ■)} 
converges weakly to the Gaussian distribution 

iv(o,/iZooM(i-ZooH)), 

where 

U^k^^{dk)Y^ 

The next theorem extends this Central Limit result to a general RRU with 
reinforcement distributions /i and v having the same mean. 
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Theorem 3.1 Assume m,, = = m > 0. Let 



H = m 




k^u{dk) + {1 - Zoo) / k^fi{dk) 



Jo 



Then, for almost every G ^2, the sequence of probability distributions 
{Kn{u!,-)} converges weakly to the Gaussian distribution 



When /i = z/, the distribution of Zoo has no point masses; this has been 
proved in [11]. May and Flournoy show in [10] that equahty of the means 
of the reinforcement distributions is a sufficient condition for proving that 
P(Zoo = {0}) = F{Zoo = {1}) = 0. As a nice consequence of Theorem \'S.1\ 
we are now able to prove that the distribution of Z^o has no point masses, 
when the means of fi and u are the same. 

Theorem 3.2 If = = m > 0, the distribution of Z^o has no point 
masses. I.e. f{Zoo = {x}) = 0, for all x G [0, 1]. 

When ^ m^,, the distribution of Z^o is the point mass at 1 or at 
0, according to whether is larger or smaller than m^; this has been 
proved in [21 [U [H] under the assumption that the supports of /i and u are 
bounded away from 0. Within the framework of the present paper, we are 
able to show that the result holds more generally when the supports of fi 
and u are contained in the interval [0,/?]. 

Theorem 3.3 Assume > my. Then ¥{Zoo = 1) = 1. 

4 Proofs and auxiliary results 

The proof of Theorem 13.11 will make use of a few auxiliary results, that we 
state and prove as Lemmas 14.11 - 14. 7[ 

From now on, given a sequence {^n} of random variables, we will denote 
by A,^„+i the increments {^n+i — C,n)- Moreover, given any two sequences 
{a„} and {6„} of real numbers, we will use the symbol a„ ^ 6„ to denote 
that ttn/bn — > 1 as — > oo. 

Lemma 4.1 Let R and D be two random variables defined on {Vt^A^P) 
with values in Br = [0,j3] and Bd = [0,oo), respectively, and let Q be a 
sub-sigma-field of A such that R is independent of Q while D is measurable 
with respect to Q. Let h be a measurable real valued function defined on 



N{Q,H{uj)Z^{uj){l- Z^{uj 



)))• 
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X Bd and such that h{-,t) is convex for all t G Bj:,. Then, for almost 
every a; G fi, 

/i(E(i?), D{uj)) < E {hiR, D)\g) H < {^h{P, D{u;)) + ^~^^^h {0, D(^))) . 

The previous inequalities are reversed if h{-,t) is concave for all t G Bo- 
Proof If vr is the probability distribution of R, 

E{h{R,D)\g){uj) = h{x,D{io))TT{dx) 
Jo 

for almost every to & Q. The left inequality is now an instance of Jensen's 
inequality. The right inequality follows after noticing that 

hix,t)<x^^ + ^hiO,t) 

for all {x,t) G Bfi x Bd, since h{-,t) is convex. □ 



As a consequence of the previous Lemma, we can bound the incre- 
ments AAn of the compensator process First note that, for all 



n = 0,l,2,.... 



almost surely, where 

Rx{n+l) Rvin+l) 



= E 



Rx(n+1) 



1 



1 



Ry jn+l) 



An I . 



(4.1) 



Lemma 4.2 If > my, for almost every u; G there is an a > such 
that 

eventually. If = niy = m > 0, for n = 0,1,2..., and almost every 
G fi, 



\ A* "^'-'^ ~ 



O 



Proof Note that t) = is a concave bounded function of re > for any 
fixed t>0. Forn = 0,1,2,..., 



= E {h{Rx{n + 1), Dn) - h{RY{n + 1), Z)„)| A); 
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hence, by applying Lemma l4.ll we get 

V "^1/ Dn{m^- my) - m^{(3 - m^) 



and 



+ /3 + Z)„ {m^ + Dn){P + Dn) 

on a set of probability one. The thesis is now a consequence of the fact that 
lim„^oo = oo almost surely (see, e.g., |10^ Proposition 2.3, Proposition 2.4]). 

□ 



Indeed, when = rriy = m > 0, two educational cases emerge by 
inspection of in the hght of Lemma 14. 1[ The first one is when /i is 
the point mass at m : then A'^_^^ > for all n = 0, 1, 2, ... and the process 
{Zn} is a bounded submartingale. On the other extreme, let /i be the 
distribution of the random variable P(, with ( distributed according to a 
Bernoulli (m//3); then < for all n = 0, 1, 2, ... and the process {Zn} 
is a bounded supermartingale. 

In pm Lemma A.l(iii)] it is proved that Mmn^oo Dn/n = m almost 
surely, when = rriy = m > 0. The next Lemma improves our general 
understanding of the growth speed of the urn size Dn- 

Lemma 4.3 Assume that min(m^, m^) > 0. For all c,a > 0, there are 
two constants < ai < a2 < oo such that 

— < E — < — 4.2 

~ (c + D„)° ~ ^ ^ 

eventually. Moreover, ifm^ = = m > 0, then 

1 1 



E- 



(c + Dn)'^ {c + Do + mn)'^ 

Proof It is trivial to prove the lemma when the supports of ^ and v are 
both bounded away from 0; this is the case, for instance, when jj, and u are 
both point masses at real numbers different from 0. For the general case set 
(T^ = min(cr^,(T3) and assume that max(o"^,cr^) > 0; without loss of generality 
we also assume that > m^ > 0. 



The left inequality in (14. 2p follows from Jensen's inequality: 
^1 1 1 

E- — > ; — > 



(c + D„)° - (c + E(D„))" - {c + Do + nmf, 
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For proving the right inequahty, we consider two cases. 
Case 1: o" > 0. For n = 1, 2, ... and i = 1, . . . , n set 

_ 5i{Rx{i) - m^) + {I - 5i){RY{i) - nil,) Ri- 

J^ni — ; S 1= • 



Then one can show that {Lni,J-ni = Ai,n = 1, 2, i = 1, . . . , n} is a martingale 
difference array such that, for all n = 1, 2, and i = 1, . . . ,n, 

\Lni\ ^ 



o^fn' 
while 

n 
i=\ 

with probability one. 

Let Sn = Y17=i ^rii- Then, by the large deviations result [IHl Theorem 1] for 
martingales and (|4.3|) we get 

^{ Er=i < nm, - an^/S) P(5„ < -n^^) ^ 
limsup —rr < hm -7— = 1, 

where $ indicates the standard normal distribution. Since 

exp(-x^/2) 
$(-x) < ' ' 

for all X > 0, we obtain 

Ri< nm^ - an^^^) < eventually. (4.4) 



i=l 



For n = 1,2, set F„ = {^^^i Ri < nvriy — un^/^}; then 

V c+Do / 

+ E ^- ■ F' 

^ nFn) , 1 



(c + Do)" (c + Z^o + ^w-z/ - cr^^/^)" 



(c + £)o + nm^)" c + ^ (l + o(l))^ 
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Hence, by ([Q]) . 

E ' 



iimsup J < 1, 



(c + Do + Tayn)'^ 
and this completes the proof of this case. 

Case 2: o" = 0. Assume o"^ > (the case it^ > is analogous). Hence v is the 
point mass at my > 0. Let (n)} be a sequence of independent and identically 
distributed random variables, independent of {An} and such that each variable 
Ryin), when multiplied by m^/my, has probability distribution equal to ji. For 
n = 1, 2, define 



infjfc: ^(5i>n}, r^ = inf|A;: ^(1 - 5^) > n}. 

't=l i=l 

By Jensen's inequality and [10^ Proposition 2.4], we then have: 

Et = EE (- V^i = k;An 

1=1 

1 " 

= EE ( r ^ V'^i = k]Ar 

V (c + A + Eti ^x(r.) + (n - I j^i 

< E E ( ; r-^ S- = k;A: 



E- 



1 

{c + Do + T^=l{^^Rx{i) + (1 - 50^y (^)))' 



Since min((T^, Var(/2y)) = (^)^o'^ > 0, Case 1 applied to a coupled RRU 
with the same initial composition and reinforcements equal to Rx{n) whenever 
5{n) = 1, and Ryin) whenever (5„ = 0, yields the thesis. □ 



Lemma 4.4 Assume = rriy = m > 0. Then 

E(^v^|AAfe|) < oo. 

fc>0 

Proof Lemma His] and Lemma IT2] vield 

VfcE(IAA^I) < ^ VfcE 2 < ^ < oo, 

^-^ ^-^ (m + JJk) ^-^ 

k>0 k>0 ^ ' k>0 

for suitable constants Ki,K2 > 0. □ 
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Lemma 4.5 

oo oo oo 

E ( Y: k'Ql) < oo, E{J2 k\QW < oo, E ( J] k^Qir) < oo. 

fc=0 k=0 k=0 

Proof For all x > and < a < 6, 

a\ 4 /a + x\4 



.bJ V 6 + X / ' 
if 0/0 is set equal to 1. Then, for k = 0,1, 

Qt = < + ' + °° )* <(! + «„ + ,3^^ ) . (45) 



It follows from Lemma 14.31 with a = 4 that 



k=Q k>0 

for a suitable constant K > 0. Hence E ( X]jt>o ^'^Qt) ^ ^'^^ '^^^ proof is similar 
for (resp. Q^): replace Rk+i with Rx{k + 1) (resp. i?y(A; + 1)) in the 
numerator of the first two terms of (14.51). □ 



The next Lemma is an auxiliary result which will be used for proving 
almost sure convergence of random series. 

Lemma 4.6 Let {ak},{bk} and {c^} be three infinite sequences of real, 
nonnegative numbers such that bk and Ck are eventually strictly positive, 
bk ~ Cfc and J2k^k/bk < oo. Then, 



dk \ ^ Ofc 

as n oo . 



bk ^ Ck 

k>n k>n 



Proof For lack of a reference, we prove the lemma. For a fixed < e < 1/2, 
let no be large enough that bk and Ck are strictly positive and (1 — e)bk < Ck < 
(1 + e)bk, for k > hq. Then, for n > no, 



k>n k>n k>n k>n k>n 

□ 

Finally, we need a general fact about convergence of random sequences; 
for lack of a better reference, see [T71 Lemma 3.2]. 
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Lemma 4.7 Let {^n} be a sequence of real random variables adapted to 
the filtration {An}- If^i^i < oo) = 1 and 

^E(^„+i|A) < oo and ^E{^l_^^\An) < oo 

n n 

almost surely, then ^„ converges almost surely. 

We can now demonstrate a proposition that will act as cornerstone for 
the proof of the main result of the paper. 

Proposition 4.1 Assume = = m > 0, and let 

Hx = m^^ E(i?^), Hy = m-'^E{R'^). 

Then 

hm nJ2iQkf = Hx, lim nJ^iQiy = Hy 

n— >oo ' ' n—>QO ' ' 

k>n k>n 

on a set of probability one. 



Proof We prove that Hm„,_»oo n ^^f^^n^Qk)"^ ~ almost surely, along the ar- 
gument used to prove Corollary 4.1 in [1]. The proof that lim^^oo n J2k>niQlk)'^ ~ 
Hy almost surely is similar and will be omitted. Let p = E(i?^). The series 

n 

converges almost surely, since it is a series of zero-mean independent random 
variables with variances bounded by n~'^f3^. This fact and Abel's Theorem 
imply that 

lim n y k-'^{R\{k -M) - p) = 0, 

k>n 

on a set of probability one. Then 



\i-m nS^ k-"^ R\{k + I) = p (4.6) 

k>n 

on a set of probability one, since lim^^oo n J2k>n ^"^ ~ 

From |10l Lemma A.l(iii)], it follows that lim.i.^^{mk)~^ Yl^^^ Ri = 1 al- 
most surely and thus 

{Q^f ^m~^k~^Rl{k + l), 
on a set of probability one. Therefore Lemma 14.61 implies that 
nY.(Qk? ~ rn-^n k-'Rj,{k + 1) 

k>n k>n 
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almost surely; however (j4.6p shows that the right term converges almost surely 
to m~'^p = Hx as n — > oo. This concludes the proof of the proposition. □ 

Proof of Theorem SH] For n = 0, 1, 2, set 

k>n 

and Wn = 'Ej{Gn\An)- Because of Lemma the process converges mono- 

tonically to zero almost surely and in , as n goes to infinity. Hence the process 
{W„}, being a non- negative super-martingale, converges to zero almost surely 
and in L"*^, as n goes to infinity. Since, for n = 1, 2, 

E(V^|Aoo - AnWAn) < E(J^ Vfc|A^fc||A) = E(G„|A), 

k>n 

almost surely, we obtain that, for all t > 0, 

n^A^ - ^.1 > t\An) < ^^^'^-^^""-'"^ < 

on a set of probability one; therefore C{'\/n\Aoa — An\\An){'^) weakly converges 
to the mass function at 0, for almost every u; € 0. Proving the theorem is 
thus equivalent to show that, for almost every w G 0, C{y/n{Mn — Moo)|^n)('^) 
weakly converges to a M{0, H{uj)), where 

H{u;) = H{uo)Z^{uj){l - Z^{uj)). 

Since {M„} is a martingale, this follows from [U Proposition 2.2] once we show 
that 

E (sup\/A?|AMfc|) < oo (4.7a) 

k 

and 

lim ny^(AMfc)^ =11 almost surely. (4-7b) 

k>n 

Proof of ()4.7ap . Since 

Vk\AMk\ < Vk\AAk\ + Vk\AZk\ 

and 

E(sup^|AAfc|) < ^ VA?E(|AAfc|), 
k 
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from Lemma 14.41 we get that 

E (sup\/A;|AMfc|) < cx) E ( sup \/A;| AZfc|) < cx). 

k k 

Note that, for n = 0, 1, 2, dn+iRxi^ + 1) = Sn+iRn+i and 



D„ D, 
1 



^nDn+l — Xn+lDn 



DnDn+1 

—4 ( XniDn + Rn+l) " (^n + <5„+li?x(n + l))Dn 

1 



XnRn+1 - Sn+iRx{n + j (4.8) 



— — — ( XnRn+1 — ^n+lRn+lDn 

Rn+l 



\,Zn — Sn+1 

J^n+1 

Qn — ( Zn — 5rH 



Dn+1 

which yields |AZ„| < Q„. Hence E ( sup^ AZfc|)^ < E(X;fcfe^Q|) < oo 
by Proposition 14.11 Since ( Esup;. \/A;|AZfc|)^ < E ( sup^ AZ^I)^ < oo this 
proves (|4.7ap . 

Proof of ()4.7bp . We spht the proof in four steps. 
First step: We show that 

hm ny^(AMjt)^ = H almost surely -<=^ lim ny^{AZk)'^ = H almost surely. 

k>n k>n 

Lemma 14.41 shows that E ( X^fe>o V^l^^fcl) < almost surely; hence 

E(^A:|A^fc|2) < oo 

k>0 

almost surely and this implies that lim^^oo Y2k>n ^l^^fcP = on a set of prob- 
ability one. However n|AAfcp < A:|A^fcp, for > n = 1,2, and thus 



lim n |AAfcp = almost surely. (4-9) 



k>r 



For n = 1,2,..., Ylik>o^'^Qk — (v^(supfc>.„ Qa:))^! and thus Lemma[13] im- 
plies that P(sup„ -y/n(supfc>„ Qfc) = oo) = which in turn implies, through 
equation (14. 8|) . that P(sup„ y^(sup;,.>„ AZ^) = oo) = 0. Hence 

lim Iny^AZfcAAfc < I sup ^/nsup AZ^ lim ^/n'S^ \AAk\ = (4.10) 

n— >oo I ^ — ' n k>n n^co ^ — ' 

k>n k>n 
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almost surely, where the last equality follows, once again, from Lemma 
Since, for n = 1,2, 

(AM„)2 = {AZn - AAnf = (AZnf + [AA^f - 2AZnAAn, (4.11) 

and imply that 

lim n V((AM„)2 - {AZ^f) = lim n V((AA)2 - 2AZ„A^„) = 

n—>oo '—^ n— +00 '—^ 

k>n k>n 

on a set of probability one. This concludes the proof of the first step. 

For the next three steps, we follow the arguments in [U Theorem 1.1] armed 
with the results provided by Proposition 14.11 and Lemma |4.5[ 

Second step: We show that 

lim n > (AZfc)^ = H almost surely <^=^ lim n > (Z^— 5fc+i)^Q| = H almost surely. 

n— ♦oo ^—^ n— >oo ^— ' 

k>n k>n 

Lemma 14.71 and (14. lip imply the almost sure convergence of ^^(AZ„,)^. Thus, 
from (j4.8p and Lemma |4.6^ we get that 

k>n k>n k+1 k>n 

as n grows to infinity; this completes the proof of the second step. 
Third step: We show that the almost sure convergence of 

m 

Y,m+i-Zk){l-Zk)HQk? (4.12) 

and 

m 

J2Hh+i-Zk)Zl{Qlf, (4.13) 

k=0 

as m grows to infinity, implies that lim^^oo n "^kyniZk — Sk+i)^Qk = H almost 
surely. 

Because of Abel's Theorem, almost sure convergence of the series ()4.12p and 
(j4.13p implies that 



lim n V(5fc+i - Zfc)(l - ZkfiQ^f = 0, 
lim nV(4+i-^fc)4(QD' = 

n — *DO ^ — * 



n— >cx> 

k>n 
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on a set of probability one. Now, from Proposition 14.11 and the almost sure 
convergence of the sequence {Zn} to Z^c, we obtain that, 



hm n VZfc(l - Zfc)2(Qf )2 = HxZoo{l - Z^f, 

n—>oo ^— ' 
k>n 

hm n ^(1 - Zk)ZiiQlf = Hy{1 - Z^)Zl 

n— >oo ^— ' 
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(4.15) 



k>n 



on a set of probability one. Equations (j4.14p - (j4.15p yield 

lim n V5fc+i(l - ZkfiQ^f = HxZ^il - Z^ 



k>n 



hm n V(l - 5k+i)Zl{Qlf = Hy{1 - Z^)Z, 

n—*oo ^— ' 



2 

oo 



k>n 



almost surely. Since, for all k > 0, Qk = Sk+iQk + (1 ~ ^k+i)Q'k ^^^d 6k+i{l 
5k+i) = 0, we have 



lim ny^iZk- 6k+ifQl 

hm 7^Y^ (5,+i(l - ZDiQff + (1 - 6,^,)ZUQl?) 

n— ►oo ^— ' \ / 



k>n 



k>n 

= HxZoo{l — Zoof + Hy{1 — Zoo)Z^ = H 
on a set of probability one. 

Fourth step: We prove the almost sure convergence of the series 

oo 

Y,m+i-Zk){l-Z,)\Q^f; 

k=0 

the proof of the almost sure convergence of '^kLo ^{^k+i — Zk)Zl{Q\)'^ is similar. 
For n = 0, 1,2, Rx{n + 1) is independent of a{5n+i-,An) and thus 



Hence, 

I E(n(5„+i - Z„)(l - Z„)2(Q^)Vn)i 



E(n(5„+i-Z„)(l-Z„)^(^(Q;^) 
< E(n|5„+i - Z„|(l - Z„)2i22^(n + 1) 
<n/32E( ^ ^ 



^^,2 R\{n+l) 



< 2nP^ 



1 

{Dn-Dor 



1 



\An) 



\An) 
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the last inequality holds because, 

1 \2 / 1 \2 {Dn+l-D^f -{Dn-Dof 



< 



Dn - Do J \Dn+l - Do J (Dn " Do)^{Dn+l " Do)^ 

2(-Dn+l — Do)Rn+l 
{Dn - Dof{Dn+l - Dof 



< 



(Dn-Do)^' 



However lim^^oo Dn/n = m almost surely, as proved in Lemma [10^ Lemma A.l(iii)]; 
thus Xln ^-^(('^"+1 ~ Zn){^ — ZnY{Qn)'^\-^n) < cxD on a Set of probability one. 
Next note that, as in [H Eq. (16)], 

E(^n2E((5„+i-Z„)2(l-Z„)4(g^)Vn)) <E(j^n2(g^)4) <oo 

n n 

because of Lemma 14.51 Therefore Lemma 14.71 implies that the series (j4.12p 
converges on a set of probability one; this concludes the proof of the fourth step 
and that of the theorem. □ 



Proof of Theorem 13.21 Recall that, if vr and vr' are probability distribution 
on M the discrepancy metric between vr' and vr is defined as 

(i^(7r',7r)= sup \ir'{B) - Tr{B)\; 

closed balls B 

this metric metrizes the weak convergence of a sequence of probability distri- 
butions {vr„} to vr, when the limiting probability distribution vr is absolutely 
continuous with respect to Lebesgue measure on M (see, e.g., [7]). 

The definition of Z^o and Theorem 13.11 imply the existence oi fl' & A such 
that P(O') = 1, and, for ah uj G Q', 

lim Zn{uj) = Zao{oj) (4.16a) 

n— >oo 

and 

lim dD{{Kn){uj)M{'^,H{uj))) = 0. (4.16b) 

n— >oo 

By way of contradiction, assume there is ap G [0, 1] such that P(Zoo = p) > 0. 
Since 

lim P(Zoo =p\An) = l|„|(^oo) 

almost surely, there is a set F G ^, F C {Z^o = p]P\^', such that P(F) > 
and, for all lo £ F, 

lim F{Z^ = p\An){uj) = 1. (4.17) 
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Fix uj G F. For n = 1, 2, set = v^(Z„(u;) — p) and consider the closed 
ball Bn = {xn}- Then, for n = 1,2, 

dD{Kn{u;),M{0,H{uj)) > \Kn{co){Bn)-M{0,H{io)){Bn)\ = Kn{u;){Bn); 

however lim,„_»oo Kn{io){Bn) = 1, because of ()4.17p . and this contradicts ()4.16bp . 

□ 




Remark 4.1 The same argument works also to show that the distribution 
of the limit composition V of the generalized Polya urns treated in ^ has no 
point masses, whenever the conditions of Theorem 1.1 in ^ are satisfied. 

Proof of Theorem 13.31 Assume > 0, otherwise it is trivial to prove that 

P(Zoo = 1) = 1. 

We will work through a coupling argument that considers two randomly 
reinforced urns with the same initial composition (x, y). Compositions of the first 
urn are described by the process {{Xn,Yn)} defined in (12. ip : the composition 
process {(X„,y„)} of the second urn is defined by 

= y„ + i?y(n + l)(l-5„+i), ■ ' 

where, for n = 0, 1, 2, ... , Ry^n + 1) = Ryin + 1) + (m^ — m^) and 6n+i is 
the indicator of the event {Un+i < + Yn)~^}. The two urns are coupled 

because the random sequences {Un} and {(14, W^)} defining their dynamics 
through equations (j2.ip and (j4.18p are the same. 

Note that = Fi{Rx{n + 1)) = E(i?y(n + 1)); hence Theorem 13.21 implies 
that the distribution of Z^o has no point masses and, in particular, 

P(Zoo = 0) = 0. 

By induction on n, we show that X„ < Xn and that Yn > Yn- For n = 
the claim is obvious because the two urns have the same initial composition. 
Assume the claim to be true for n. Then: 

Xn +Yn Xn+ Yn {Xn + K„) (X„ + K„) 

which implies 6n+i > Sn+i- Hence 

Xn+l - Xn+l = {Xn " Xn) + Rx{n + l){5n+l " dn+l) > 0, 
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and 



Yn+l - Yn+l = {Yn - Y^) + Ryin + l){5n+l - 5n+l) 

+ (m^ - my){l - 5n+i) > 0. 

Therefore eq. ()i39]l holds for all n; hence P(Zoo = 0)<P(Zoo = 0) = 0. 

What remains to prove is that P(.^oo G (0,1)) = 0. To get this, we can 
use the same argument as in |141 Theorem 5], once it has been proved that [14^ 
Eq. (11) in the Proof of Lemma 4] holds without the assumption of boundedness 
away from for the supports of the reinforcement distributions. Defining A* as 
in equation (|4.1|) . this is tantamount to show that 



on a set of probability one. However, when > rriy, Lemma [4.2l shows that, for 
almost every a; G il, there is a > such that A*^{ijj) > a/{Do + n/3) eventually; 
hence (I4.20p is true. 



5 A final remark on absolute continuity 

Having proved that the distribution of the hmit proportion Zoo of a RRU 
has no point masses, when the means of the reinforcement distributions 
fi and u are the same, the next obvious question concerns its absolute 
continuity with respect to Lebesgue measure. Theorem 13.21 implies that 
P(Zoo e S") = 0, for all countable sets S in [0, 1]. The next step would be 
to show that, if 5* is a Lebesgue null set, then F{Zoo G 5) = 0. Unfortu- 
nately, the idea developed in the proof of Theorem 13.21 cannot be furtherly 
exploited to produce such result. In any case, if the closed balls Bn appear- 
ing in the proof are replaced with the "holes" of a porous set (for the link 
between a-porous sets and measures, see [13], El]), it is possible to show 
that 



for all (T-porous sets S in [0, 1]. Unfortunately this is not enough to prove 
that the distribution of Z^o is absolutely continuous; indeed T-measures 
are singular with respect to the Lebesgue measure, but they attribute 0- 
measure to any cr-porous set (see, e.g., [TSj 120]). 



n 




(4.20) 



k=l 



□ 



p(Zoo eS) = o, 
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